An active pixel sensor (“APS”) is a special kind of light sensing device. Each active pixel includes a light sensing element and one or more active transistors within the pixel itself. The active transistors amplify and buffer the signals generated by the light sensing elements in the pixels. One type of such APS devices is disclosed in U.S. Pat. No. 5,471,515 by Fossum et al., the disclosure of which is incorporated herein by reference.
APS devices represent an emerging technology in a wide range of imaging applications. APS has a number of significant advantages in comparison with the well-developed and widely used charge coupled devices (CCDs). Because of the number of stages in a CCD, a nearly perfect charge transfer efficiency must be maintained in a CCD in order to maintain good signal fidelity. This usually makes CCDs susceptible to damage from both ionizing and displacement damage. In particular, the displacement damage caused by high-energy particles and photons may prove to be deadly to CCDs.
Large CCD arrays present engineering difficulties in manufacturing. Device miniaturization is difficult with CCD devices since the extreme requirements of CCDs necessitate a special formation process. This formation process prevents CCDs from being easily integrated with on-chip integrated circuits such as complementary-metal-oxide-semiconductor (CMOS) integrated circuits. CCDs also suffers limited spectral responsivity range and readout rates. Furthermore, the readout of CCDs is destructive, i.e., their stored value is destroyed upon reading out. Implementation of the nondestructive readout in a CCD device is difficult.
In contrast, an APS device receives and processes input signals with the active pixel itself, thus eliminating the charge transfer over distances that are inherent in CCDs. Consequently, many drawbacks associated with CCDs are avoided in APS devices. For example, the performance of APS devices can be maintained as the array size increases. The APS readout rate is usually higher than that of CCDs. Since CMOS circuitry is often associated with the image sensor, the power consumption can be significantly reduced. APS devices are inherently compatible with CMOS processes, allowing reduced cost of manufacturing. Many on-chip operations and controls can be relatively easily implemented including timing and analog-to-digital conversion. APS devices are also less vulnerable to radiation damage and can be designed for non-destructive readout. Moreover, the active pixels of APS devices allow random access and on-chip signal processing.
One important benchmark in performance of imaging devices is the ratio of the saturation level of the detectors and the noise level thereof or the signal-to-noise ratio (SNR). This can be expressed in terms of dynamic range of the device. The dynamic range is usually expressed in dB by 20 log(SNR) or in binary (bits) by log2(SNR). The larger the dynamic range, the better an imaging devices.
In particular, a large dynamic range is desirable in applications for sensing low light signals and capturing images with large variations in brightness.
The dynamic range of previously-reported CMOS-based APS circuits has been limited by both the saturation level of the signal chain circuit, which is typically about 1.2 volts for a 5-volt power supply, and the noise floor of the sensor, which is typically about 150 μV. This results in a dynamic range of approximately 78 dB (13 bits), which is comparable to the dynamic range of the state-of-art CCD devices.
The output voltage signal of an imaging device is dependent on the input light level, the efficiency of the optical coupling device and the detector characteristics including the quantum efficiency, the effective active sensing area, the integration time, and the electrical conversion gain of volts/electron. The output signal can be approximately expressed as the following:
                                          V            out                    ≈                                    Φ                              4                ⁢                                  f                  2                                                      ⁢                          τ              optics                        ⁢                          A              det                        ⁢                          T              int                        ⁢            ηG                          ,                            (        1        )            where Φ is the incident photon flux, f is the f-stop of the coupling optical system, τoptics is the transmission of the optical system, Adet is the pixel size, Tint is the integration time, η is the pixel quantum efficiency, and G is the conversion gain in volts/electron. The typical values for a APS device are f/8, 80% for τoptics, 20 μm for Adet, 33 ms for Tint, 25% in η, and G˜10 μV/e−, respectively.
The exposure of an imaging device for a given hardware configuration is usually controlled by changing either the aperture (i.e., f-stop) or the integration time. If the input light level is low, the integration time is increased to improve the signal-to-noise ratio. If the input light is bright, the integration time is reduced to avoid saturation of the detector while maintaining a high SNR. The lower limit of the integration time is set by the readout time.
A number of prior-art techniques exist for controlling the integration time. For example, the integration time of a CMOS APS device with N rows of pixels can be controlled by resetting a row of pixels in advance of readout. If the readout time for one row of pixels is Trow, the total frame readout time is NTrow. Since the pixels of a row are reset for a new integration upon readout, the integration time is simply the frame time NTrow.
FIG. 1 illustrates an exemplary APS. A pixel array 102 has N rows of pixels with each row having M columns. A column-parallel signal chain 104 such as a sampling capacitor bank with M storage cells is used for readout.
In a specific operation of readout, a particular row is selected for readout at one time. The sensor data from the M pixels in the selected row is copied onto the capacitor bank 104. The copy process also resets the pixels in the selected row and begins a new integration. The M storage cells in the capacitor bank 104 is then scanned sequentially for readout. The above readout process is then repeated for the next row. Therefore, the integration time for each pixel is identical and is equal to the readout time of a frame, i.e., NTrow.
Another prior-art approach uses an electronic switching mechanism built in the pixel design of a CMOS APS device, allowing electronic shuttering and simultaneous integration. This was disclosed in U.S. Provisional Application No. 60/010,305 filed on Jan. 22, 1996 by Fossum et al.
The inventors of the present invention recognized a limitation of the above two methods in that the exposure remains the same for the entire image. This can result in distortion of the detected images in some circumstances. For example, an outdoor scene might be optimally exposed for the sunlit areas but underexposed in the shadowed areas. This can result in loss of detail in shadowed areas.
One solution to the above problem is to use a nonlinear output sensor. Such a sensor is designed to have a high differential gain for low light levels, and low differential gain for high light levels. It is desirable that the nonlinear gain be achieved within each pixel. As a result, pixel to pixel variations in the transfer function can lead to unacceptable values of fixed pattern noise.
Another approach is to implement multiple storage sites in each pixel in CCD devices to record different integration signals from the same pixel. In case of two storage sites in each pixel, one storage site corresponds to a short integration period and the second to a long integration period, thus increasing the dynamic range. The readout data can be transformed to a lower bit count encoding using nonlinear techniques for display, storage, and transmission. One limitation of this technique is a decreased readout rate since the readout time may be doubled since twice as much of the amount of data may need to be read. Another limitation is the reduced fill factor since two storage sites occupy considerable pixel area.
In view of the above limitations, the present invention describes a new technique in APS sensors and operation methods thereof to increase their dynamic ranges. According to the preferred embodiments of the present invention, at least two different integration times are obtained for each active pixel. One or more column-parallel signal chains are used to implement the multiple integration times for each active pixel. For an APS device with only one column-parallel signal chain for readout, each pixel is sampled multiple times during a single frame readout, thus resulting in multiple integration times.
Another aspect of the present invention increases the frame rate by using more than one column-parallel signal chain to achieve multiple data points of different integration times for each pixel. A maximal frame rate is achieved for a given APS device by having the number of column-parallel signal chains match the number of different integration times. A preferred operation mode of such system in accordance with the present invention includes simultaneously copying multiple columns into the column-parallel signal chains and subsequently reading out all the column-parallel signal chains simultaneously with each being scanned in a sequential manner. The frame rate is increased by a factor equal to the number of the column-parallel signal chains as compared with achieving the same number of integration times with a single column-parallel signal chain for readout. The dynamic range of the APS device is thus extended by a factor given by the ratio of the longest integration time versus the shortest integration time.
A new readout method is also described. This method can be used to improve the dynamic range of a conventional APS device having a single column-parallel signal chain by achieving multiple integration times for each active pixel. Special advantages are obtained when used with non-destructive readout of an APS device in achieving multiple integration times of each pixel.
Another aspect incorporates digital processors including analog-to-digital converters in the column-parallel signal chains.
On-chip buffer memory units and/or multiple source followers may be used for each pixel to further enhance the flexibility and performance of the systems in accordance with the present invention.